DSpot: Test Amplification for Automatic Assessment of Computational Diversity

In this work, we characterize a new form of software diversity: the existence of a set of variants that (i) all share the same API, (ii) all behave the same according to an inputoutput based specification and (iii) exhibit observable differences when they run outside the specified input space. We quantify computational diversity as the dissimilarity between execution traces on inputs that are outside the specified domain. Our technique relies on test amplification. We propose source code transformations on test cases to explore the input domain and systematically sense the observation domain. We run our experiments on 472 variants of 7 classes from open-source, large and thoroughly tested Java classes. Our test amplification multiplies by ten the number of input points in the test suite and is effective at detecting software diversity.